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Executive Summary 



Purpose 



As a nation, we continue to spend billions of dollars to discover, develop, 
test, and refine new medical technologies. However, little effort is 
exerted to determine if these technologies, once they are ready for pub- 
lic use, realize the potential they displayed during their development. 
The purpose of this report is to examine the extent to which one 
advance in the treatment of breast cancer has benefited patients. This 
report responds to a request by the Subcommittee on Health and the 
Environment of the House Committee on Energy and Commerce that gao 
examine the issue of cancer patient care. 



Background 



In the mid-1970's, great excitement was generated by reports from two 
separate clinical trials that chemotherapy administered following sur- 
gery (ac^juvant chemotherapy) was beneficial for premenopausal women 
with breast cancer that had spread to the lymph nodes under the arm. 
Subsequent to these reports, the use of adjuvant chemotherapy 
increased considerably for this group of patients. 



The results of continued experimentation on the benefits of ac^uvant 
chemotherapy led to a consensus that **a(ijuvant chemotherapy has 
demonstrated a significant reduction in mortality in premenopausal 
women with histologically positive axillary lymph nodes.'' In light of 
this consensus that at^uvant chemotherapy can increase survival, and 
given the increased use of this therapy, logic would indicate that the 
survival of premenopausal, node-positive breast cancer patients should 
have improved since the introduction of ac^uvant chemotherapy. The 
issue of how the survival of breast cancer patients has changed over 
time is the focus of this review. 



Rpmilt^ in Rripf Despite a considerable increase in the use of chemotherapy since 1975, 

ivcoUltD 111 DL Icl there has been no detectable increase in survival for the patients who 

should have benefited most from the advent of this therapy. Although 
this finding could be interpreted as evidence that chemotherapy is not 
beneficial, such an interpretation is unlikely to be correct. Controlled, 
prospectively designed, clinical studies have been conducted and have 
shown that chemotherapy does extend survival for specific types of 
breast cancer patients. When these studies were critically examined by 
* cancer experts around the world, a consensus was reached that aclju- 

vant chemotherapy improves the survival of premenopausal, node-posi- 
tive breast cancer patients, gao's work does not contradict these 
findings. What it does shew is that there seem to have been problems in 
moving the treatment for breast cancer from the laboratory to the 
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patients, gao believes that the issue of the survivability benefits from 
postsurgery chemotherapy treatments needs further study. 



Principal Findings 



Table 1: Premenopausal, Node-Positive, 
Stage II Breast Cancer Patients: 
Treatment and Survival Patterns 



The accompanying table shows the treatment and survival patterns for 
the group of patients who should have benefited most from ac^uvant 
chemotherapy. 



Year of diagnosis 


Percent receiving 




Survival rate' 




chemotherapy 


3-year 


5-year 


7-year 


1975 


23% 


.82 


.72 


.64 


1976 


45 


85 


.71 


.64 


1977 


46 


.83 


.71 


64 


1978 


53 


.83 


70 


.63 


1979 


55 


.83 


.71 


.65 


1980 


62 


.86 


.77 




1981 


66 


83 


.72 




1982 


72 


.84 






1983 


69 


.85 







^The survival rales provided assume complete follow-up only through the end of 1985 That is why the 
last year for 3-year survival ;s 1983. for 5-year, 1981 . and for 7-year, 1979. 

When these data are subjected to statistical tests, they show no statisti- 
cally significant improvement in patients' survival. This finding was 
consistent across all three analytic methods gag employed to detect 
changes in survival, gag concludes that the leek of a detectable improve- 
ment in patients' survival may result from one or a combination of the 
following: 

Many patients still do not receive adjuvant chemotherapy. 

The benefits of chemotherapy are small and therefore difficult to detect. 

There are problems with how well the treatments are implemented. 

One additional finding was that the survival of women diagnosed in 
1980 was greater than that of women diagnosed in any other year, gao 
could find no exr-^ ' ^ * for this finding. 
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Recommendation 



GAO recommends that the secretary of the Department of Health and 
Human Services (hhs) initiate a study to detennine why there has been 
no visible improvement in the survival of pre.nenopausal, node-positive 
breast cancer patients despite the advent of ac^uvant chemotherapy. 
Given the methodological obstacles involved in such a study, the secre- 
tary should seek expert advice in assessing the feasibility of conducting 
the recommended research and in developing the study design most 
likely to succeed. (See pp. 26-27.) 



Agency Comments 



In HHS's response to a draft of this report, it concurred with gao's recom- 
mendation, suggesting a small modification with which gao concurs. The 
recommendation as stated above incorporates hhs's views. 

The most pervasive concern expressed by hhs in its written comments 
was that the study design gao used had low statistical power, hhs 
pointed out that this is the case because the majcimum benefit that can 
be derived from ac^juvant chemotherapy is "modest," ranging from 7.3 
to 10.8 percentage points. However, gao's study was begun on the basis 
of HHS's response to a 1987 gao report in which hhs argued that there 
was a "confirmed 25 percent improved survival for Stage II preme- 
nopausal women treated with a^uvant chemotherapy." If this 25-per- 
cent figure is used, the power of gao's study is adequate. 

The fact that hhs has presented conflicting estimates of the expected 
benefits of ac^juvant chemotherapy suggests the need for further study 
of what has happened in the treatment of breast cancer patients. Fur- 
ther bolstering gao's recommendation is ims's agreement that large num- 
bers of patients still do not receive ac^juvant chemotherapy and that 
problems with how well the treatments are implemented may contribute 
to the absence of a detectable improvement in sur/ival among preme- 
nopausal Stage II breast cancer patients. 
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Adjuvant Chemotherapy and Breast Cancer 



Introduction whether a new treatment has the potential to benefit patients is a ques- 

tion that is best answered under controlled, experimental conditions. An 
equally important question, and one that cannot be answered through 
experiments, is whether the treatment actually benefits patients in the 
**real world" once the experiments have been concluded. Our objective in 
this report is to determine, for one specific medical advance, whether its 
potential to extend patient survival has been realized. 



This report responds to a request from the Subcommittee on Health and 
the Environment of the House Committee on Energy and Commerce that 
we examine the issue of cancer patient care. The specific focus of our 
work is on the advance made when it was discovered that administering 
chemotherapy following surgery (adijuvant chemotherapy) improves the 
survival chances of premenopausal breast cancer patients. In the 
remainder of this chapter, we give a brief historical overview of this 
discovery. 



Until the 1960's, the only two options available for effectively treating 
most forms of cancer were surgery and radiation therapy. The benefits 
and limitations of surgery were clear-cut. If the operation could remove 
all the cancerous cells from the body, the patient was often cured. If not, 
the patient invariably died. Although radiation therapy expandea the 
physician's ability to reach and kill cancer cells, it too had the same 
basic limitation as surgery. That is, both treatments were effective only 
for achieving local control. Once cancer cells had spread through the 
body by the process of metastasis, neither surgery nor radiation therapy 
could offer much hope of cure. 

All this changed with the introduction of chemot"ierapy to the 
armamentarium of cancer care. Chemotherapy — that is, treatments that 
use drugs to **poison" cancer cells — allowed cancer to be treated for the 
first time as a systemic disease. Using drug;s that spread throughout the 
body, cancerous deposits could be attacked wherever they were located. 

Although it was known in the 1920's that certain drugs could kill cancer 
cells, it was not until the late 1950's that researchers were able to 
develop a practical treatment for one relatively rare form of cancer. 
This was followed by the determination in the mid4960's that a combi- 
nation of drugs cou!d be successfully used to treat acute lymphocytic 
leukenria, the most common form of childhood leukemia. Since then, dif- 
ferent combinations f chemotherapy have been shown to work against 
many other forms of cancer. 



The Development of 

Ac^uvant 

Chemotherapy 



ERLC 
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For the most pait, however, until the mid-1970's, drugs showed little 
promise in treating carcinomas, the "solid tumors" that account for 
approximately 85 percent of all cancer cases in this country. Then there 
was Sv.mft promising news. Two articles appeared within a year of each 
other, de x-ribing the results of experiments on the use of ac^juvant 
chemotherapy to treat breast cancer patients. The first of the articles, in 
1975, was from the National Surgical Adjuvant Breast Project (nsabp) 
study that compared surgery patients receiving L-phenylalaiiine mus- 
tai-d (IrPAM) after their operations to patients receiving no additional 
therapy after surgery.* Prelixainary results from this trial showed that 
patients treated with l-pam had longer **disease-free survival" than their 
counterparts, who had only surgery as treatment.^ Little more than a 
year later, even more exciting news came from an experiment conducted 
in Italy. Researchers at the Milan Tumor Institute reported that breast 
cancer patients treated with a combination of three drugs following sur- 
gery (cyclophosphamide, methotrexate, and 5-fluorouracil, or cmf) had 
one fourth the recurrence rate that women had whose treatment 
included only surgery.-' 

The immediate reaction to the article describing the Italian study was 
considerable. In an editorial appearing in the same journal issue, the 
work was described as being of "monumental importance" and the find- 
ings were characterized as "nothing short of spectacular."^ More impor- 
tantly, the percentage of premenopausal, nodepositwe patients 
receiving chemotherapy almost doubled in 1976 and continued to rise 
until 1982. No doubt much of this positive reaction stemmed from the 
fact that chemotherapy had been finally shown to have some erfect, in a 
clinical setting, against a prevalent form of cancer. 

What was perhaps missed in the initial assessment of the acy'uvant ther- 
apy trials was that the results were based on a very short observation 
period. In fact, the length of follow-up in the Italian study was so short 
that the researchers who reported the results warned that their "results 
should be considerer^ with caution, the effect on survival not being 



'B. Fisher et al.. **LrPhenylalanine Mustard (I^PAM) in the Management of Primary Breast Cancer: A 
Report of Early Findings," New England Journal of Medicine . 292:3(1975), 117-22. 

-"Disease-free survival" is defined as the length of time that passes between the date of diagnosis and 
either recurrence or the end of an observation period. 

•^G. Bonadonna et al., "Combination Chemotherapy as an Actjuvant Treatment in Operable Breast 
Cancer," New England Journal of Medicine . 294:8 (1976), 405-10. 

(1976)1^^^^^ Advance in Breast Cai.^er Therapy," New England Journal of Medicin e. 294:8 
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known." This warning turned out to be proplietic. Once a sufficiently 
long period had passed in which to evaluate the ben^^fits of chemother- 
apy on extending overall survival (as opposed to reducing recurrences), 
the results in both the Milan and nsabp studies were found to be less 
dramatic. The **finar' results showed a benefit for chemotherapy among 
all premenopausal, node-positive patients in the Milan trial and a more 
limited benefit in the nsabp study (only premenopausal patients with 
fewer than four positive nodes). 

As patients were observed for longer periods of time, the shifting results 
of acyuvant chemotherapy trials led to both debate on the efficacy of 
drugs for treating breai^c cancer and further experimentation.'' The 
NSABP continued in its plans to add other drugs to l-pam to see if combi- 
nations of drugs would be more effective than single agents.'' The 
researchers in Milan compared different durations of therapy (6 versus 
12 cycles) to one another." New drugs were tried, as were combinations 
of drugs with establish^^d or new modalities of treatment.^ Research was 
conducted both here and abroad.'^ 

In response to the growing body of knowledge developed from these tri- 
als, the National Institutes of Health convened experts from around the 
worla to see if there was agreement on the benefits of chemotherapy. At 
this 1985 meeting, a consensus was reached that 

"adjuvant chemotherapy has demonstrated a highly significant increase in disccise- 
free survival and a significant reduction m mortality in premenopausal women with 



'For an illustration of the debate, see S.H. Levitt and R.A. Polish, *Tne Case for Actjuvant CMF Chem- 
otherapy in Breast Cancer: Has It Been Made?" Cancer Clinical Tnals , 4 (1981), 363-64, 

^'B. Fisher et al., "Ac^jiivant Chemotherapy for Breast Cancer: An Overview of NSABP Fmdings," 
International Advances in Surgical Oncology , vol. 5 (New York: Alan R. Jss, Inc., 1982), pp. 65-90. 

'G. Bonadonna et al., "AcUuvant Chemotherapy Trials in Resectable Breast Cancer with Positive Axil- 
lary Lymph Nodes; The Experience of the Milan Cancer Institute," in S.E. Jones and S.E. Salmon 
(eds.), AcUuvant Therapy of Cancer IV (Orlando, Fla.: Gnine & Stratton, 1984), pp. 196-207. 

^J.M. Morrison et al., "The West Midlands Trial of AcUuvant Chemotherapy for Axillary Node Posi- 
tive Breast Cancer: A Controlled Cllnicai Trisi " in Jones and Salmon, pp. 253-59; D.C. Smith et al., 
"AcUuvant Radiotherapy Chemotherapy in Breast Cancer,** in Jones and Salmon, pp. 283-89; T.B. 
Hakes et al, "CMF +/- Levamisole Breast AcUuvant Chemotherapy: 5*year Analysis," Proceedings of 
the American Society of Clinical Oncology , 1:83(1982). 

•'The NSABP studies are one example of U.S. research. The West MidlanQ3 trial was conducted in 
England and the onginal CMF trial in Italy, to name but two of t.ie foreign studies. 
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histolog. tlly positive axillary lymph nodes. Acljuvant chemotherapy can now be 
considered standard care for these patients."'^' 

This consensus statement shows that the question of whether chemo- 
therapy has the potential to extend patient survival has been settled. 
What has not yet been resolved is whether the actual use of chemother- 
apy has realized this potential 

I ^ 

Report Overview ^^^^ chapter, we describe how our study was conducted. In that 

context, we describe the data we used, viiy we focused on acUuvant 
chemotherapy for breast cancer, how patients were selected for inclu- 
sion in the study, how "benefit" is defined, and the analyses that were 
performed to develop and support our findings. Our results a ^e pre- 
sented in chapter 3, the concluding chapter of the report. A discussion of 
statistical issues is provided in appendix I and the patient selection cri- 
teria are described in appendix II. The comments of the Department of 
Health ana Human Services (hhs) are in appendix III. Appendix IV lists 
the mgjor contributors to this report. 



^National Institutes of Health, Cons<»nsus Development Conference Statement, AcUuvant Chemother- 
apy for Breast Cancer, 5:12 (1986). 
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Chapter 2 



Objective, Scope, and Methodology 



The objective of this study is to determine whether there has been any 
detectable change in survival for the group of breast cancer patients 
who should have benefited the most from the increasing use of ac^uvant 
chemotherapy. The scope of the project was largely defined by the 
availability of data, the advances in treatment that have been made in 
recent decades, and the population of patients for whom those advances 
were relevant. Each of these dimensions is discussed below, followed by 
descriptions of how "benefits'' Lie measured and how the analyses that 
support our findings were conducted. 



Given our objective of determining whether there was any change in 
survival for a specific group of patients, we needed a data set that con- 
tained information on the characteristics of both the disease and 
patients and that tracked patients over time. The Surveillance, Epidemi- 
otogy, and End Results (seer) program, initiated by the National Cancer 
Institute (NCI) in 1972 and currently the primary source for data on can- 
cer incidence and patient survival, provided us with the needed data. 

Twice a year, seer receives information on incidence and follow-up for 
cancer patients from population-based cancer registries in the United 
States and Puerto Rico. Together, these registries cover 12 percent of 
the total population. The population covered by seer is not a probability 
sample of the country, but the data are believed to represent overall 
cancer patterns. 

The SEER data base contains information at the case level on the type of 
cancer, the date of diagnosis, how far the disease had advanced when it 
was discovered, how the patient was treated, the most recent date that 
contact was made with the patient, and whether the patient was alive or 
dead on that date. In addition, seer also provides information on 
patients' characteristics (age, race, and sex) that is useful for creating 
homogeneous strata of patients for analysis. 

When we began our study, the last full annual cohort of patients in seer 
were those who were diagnosed during 1985. For this reason, we used 
that year as the cutoff for entering patients into our study. Our decision 
to restrict entry at the other end to patients diagnosed after 1974 was 
based on the fact that the treatment advance (adjuvant chemotherapy) 
was made in 1975. Since follow-up data for patients were available only 
through 1986, this meant that for patients diagnosed in later years, only 
short-term survival could be computed (for example, only 1-year sur- 
vival was available for patients diagnosed in 1985). 
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In our earlier study, in which we examined the use of "breakthrough" 
treatments, we required criteria to decide which advances in treatment 
should be included.' We established three criteria: 

1. the treatment had been proven to increase patients' survival in a large 
randomized clinical trial (necessary to ensure that benefits of the 
advance were measurable), 

2. the results of that trial had been published by 1982 (so as to allow us 
to determine patterns of use with the available data on treatment), and 

3. the treatment was relevant for an identifiable group of cancer 
patients (so that we could include in our analyses only the patients who . 
should have benefited from treatment). 

As we began our first study, we asked the assistance of nci in determin- 
ing the therapies that met all three requirements. In response to our 
request, nci forwarded a list of treatments, seven of which were 
included in our report.- 

In the present study, the focus is on the actual benefits of the new thera- 
pies to patients. The analyses we used to determine benefit required 
that three additional criteria be satisfied for any therapy to be appropri- 
ate for inclusion. One of these was that the therapy be relevant for a 
large enough number of patients to ensure that the estimates of benefits 
not be overly subject to random fluctuations as a consequence of small 
sample size. The second criterion was that there be no known change in 
the prognosis of patients that was unrelated to treatment. Any such 
change would make it impossible to determine why patient survival had 
(or had not) improved over time. The final criterion was that there be a 
considerable increase in the frequency with which the treatment 
advance was given to patients. Such an increase clearly had to have 
occurred before treatment-related survival gains could be expected. 



Criteria for Selecting 
Acyuvant 
Chemotherapy for 
Breast Cancer 



■ U.S. General Accounting Office, Cancer Treatment 1975-85: The Use of Breakthrough Treatments 
for Seven Types of Cancer , GA0/™mu-88-12BR (Washington, D.C.: January 1988)7 

-The seven were ac^juvant chemotherapy for breast cancer and colon cancer, ac^uvant radiation ther- 
apy for rectum cancer, and chemotherapy for small-cell lung cancer, testicular cancer. Hodgkin's dis- 
ease, and non-Hodgkin's lymphoma. 



RIC 
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The only therapy that met the three criteria added for this study was 
ac^uvant chemotherapy for the treatment of premenopausal breast can- 
cer patients.'^ No apparent change in prognostic factors for this group of 
patients has been noted, and, since 1975, there has been a considerable 
mcrease in the frequency with which the therapy is given. 

It seemed important to examine improvements in breast cancer patients' 
survival for two reasons. First, the incidence of the disease is on the 
rise, as is the mortality rate, among premenopausal women. Second, the 
recent decision by nci to issue a clinical alert recommending that ac^ju- 
vant chemotherapy also be considered for all node-negative patients 
focused new attention on the question of the extent to which node-posi- 
tive breast cancer patients have actually benefited from such therapy. 



The Selection of 
Patients 



Few if any treatments are appropriate for all patients suffering from 
any one type of cancer. As a disease progresses, the appropriate treat- 
ment for it typically changes. Given that our goal was to determine 
whether breast cancer patients benefited from the advent of ac^juvant 
chemotherapy, it was reasonable to focus on the patients for whom the 
treatment had been proven effective. As mentioned above, the initial 
clinical trials showed chemotherapy to be effective only for premen- 
opausal, node-positive patients. For this reason (and at the suggestion of 
NQ stafO we included in our analyses only breast cancer patients 50 
years of age or younger at time of diagnosis (as a surrogate for meno- 
pausal status) who were node-positive and who did not have any metas- 
tases to distant sites.^ Using these criteria, as well as a requirement that 
a tumor not exceed 5 centimeters in size upon diagnosis, we selected all 
ihe brr-dst cancer patients in the seer data base who satisfied our crite- 
ria and were diagnosed after 1974. 



Measuring Benefits 



Progress against disease can come in many different forms, including 
new or improved treatments. When a new treatment is developed, it can 



•^The treatments for prostate, colon, and rectum cancer had not been proven to extend survival in 
tnals concluded by 1982. The treatments for osteosarcoma and soft-tissue sarcoma would not yield 
large enough samples of patients. The treatments for the two lymphomas and testicular cancer 
showed no increased use over the 1 1-year period for which data were available. Finally, a data prob- 
lem with small-cell lung cancer made patients diagnosed after 1982 noncomparable with patients 
whose cancers had been detected by then. The specific problem was a change in how the extent of 
disease was coded in SEER data. 

■^The age cutoff of 50 is not a perfect surrogate for menopausal status because the onset of meno- 
pause may be earlier or later and some women have hysterectomies that induce early menopause. 
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be seen as beneficial for different reasons. For example, the treatment 
may increase cure rates, reduce treatment-associated side effects, 
reduce cost, or increJise the length of time until relapse. Which of these 
benefits is the most "important'' is impossible to say, and it is likely that 
the value of each one is weighed differently by patients, physicians, 
researchers, and health insurance providers. The benefits also differ 
considerably in the ease with which they can be measured. For this 
study, we selected overall survival as the benefit of interest.'^ 

Three factors support our decision to focus on overall survival. First 
and foremost, whether a patient lives and, if so, for how long are clearly 
important considerations. Second, measuring overall survival is more 
feasible than measuring more abstract concepts such as "quality of life.'' 
Finally, the clinical trials that showed ac^uvant chemotherapy to be ben- 
eficial for young bvedst cancer patients based that conclusion on the 
ability of chemotherapy to extend life. 

Our focus on overall survival, while appropriate, also means that our 
conclusions on the benefits of therapy should not be extended to reach 
conclusions on such issues as whether breast cancer patients live health- 
ier or happier lives than they used to. 

Analysis Plan ^^^^^ underlying our analyses rests on two facts: chemotherapy has 

^ been shown to be effective in extending the lives of premenopausal, 

node-positive breast cancer patients, and there has been a dramatic, 
twofold to threefold increase in the use of chemotherapy for this class 
of patients in recent years.<* From these facts, it follows that one should 
be able to observe some improvement in the survival experiences of 
breast cancer patients. To test whether this actually occurred, we 
employed three different statistical procedures. Each of the procedures 
involved making comparisons between diagnostic cohorts (that is, 
cohorts that are defined by the year in which the patients were diag- 
nosed as having cancer). 



'Overall survival is defined as the time that passes between the date of diagnosis (starting point) and 
either death or the end of data collection (ending point), whichever comes first. 

'The ambiguity regarding the magnitude of the increase is a function of how one enumerates the 
patients for whom chemotherapy was planned but for whom SEER does not indicate whether it was 
given. If these patients are counted as getting chemotherapy, the threefold increase is the more accu- 
rate figure. 
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One approach we took was to compare observed survival rates of each 
successive diagnostic cohorts Despite the frequency with which sur- 
vival rates are used, one problem with them is that they take measure- 
ments only at specific points and not across entire intervals. Figure 2.1 
shows the erroneous conclusions that can be reached by examining only 
survival rates. 



Figure 2.1: Hypothetical Survivs; Z\ir:^9 ■■■B^^^^H 

P«rc«nt of PaftlMTts Surviving 




Ymts Following DIagnotis 



Group A 
■■■■ Group B 

As can be seen from this figure, the percentage of group A that remains 
alive is always greater than the percentage of group B, except at the 5- 
year point. If we compared only 5-year survival rates, both groups 
would look the same, even though they have different survival profiles. 
Examining survival rates at more than one point (for example at 3, 5, 
and 7 years) reduces the likelihood of making this type of error but does 
not eliminate it. 



^The observed survival rate is defined as the percentage of patients who survive for a specified 
period of time (for example, 5 years) from the time of diagnosis, 
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The second set of analyses we performed avoids the limitations of 
"point comparisons.'' In these analyses, we compared the survival erpe- 
riences of women diagnosed in 1975 with those diagnosed in 1976, those 
diagnosed in 1976 with those diagnosed in 1977, and so on, through 
1984-85, using the lifetable method .^ This procedure compares actual 
length of survival (not only percentage surviving at specified times) of 
all cases across groups and provides statistical tests that indicate 
whether survival is different between the groups. We employed the 
LIFETEST procedure that is available on the SAS software program to 
perform these analyses. In evaluating the output from these computer 
runs, our principal focus was on the significance levels achieved in the 
Generalized Wilcoxon and Logrank tests.'' 

Because of the possibility that the populations we compared changed in 
some way and that, if such changes occurred, they might be related to 
survival, we also analyzed the data using a procedure known as propor- 
tional hazard modeling.'" For this analysis, also known as Cox regres- 
sion, we simultaneously entered four variables that identified the age 
and race of the patient, the size of the tumor, and how far the tumor had 
extended at the time of diagnosis, as well as 10 dummy variables denot- 
ing the year in which the cancer was diagnosed. In reviewing the results 
of this analysis, our focus was on whether the Beta coefficients associ- 
ated with the year of diagnosis dummy variables achieved statistical 
significance." Significant coefficients wouM mean that any change in 
the risk of dying was probably **rear' (not a function of chance). If the 
coefficients did not achieve significance, we concluded that any 
observed change was probably the result of chance. 



^Elisa T. Grant, Statistical Methods for Survival Data Analysis (Belmort, Calif.: Wadsworth, Inc., 
1980), ch. 5, and David Oakes, "Survival Analysis," European Journal of Operations Resea rch, vol. 12 
(1983), 3-14. ' 

'^Significance level is defined as the probability that any observed difference is simply the result of 
chance. 

''*D.M. Finkelstein and R.A. Wolfe, "Methods of Survival Analysis," in R.G. Cornell (ed.). Statistica l 
Methods for Cancer Studies (New York: Marcel Dekker Inc., 1984), pp. 147-52. 

' 'The Beta coefficients (which actually denote a hazard function) can be transformed (by exponentia- 
tion) to provide a measure known as the nJative risk . If the relative nsk is greater than 1 .0, then 
there is an increased hazard, and if less than 1 .0, a decreased hazard. 
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Our findings also include data on patterns of use for ai^uvant chemo- 
therapy. These are drawn from our January 1988 report, to which read- 
ers are referred for a description of how that analysis was performed.'- 

Our work was performed in accordance with generally accepted govern- 
ment auditing standards. Howtv^er, it should be noted that we did not 
verify the seer data provided to us by nci. 



'-U.S. General Accounting Office> Cancer Treatment 1975-85: The Use of Breakthrough Treatments 
for Seven Types of Cancer , GAO/PEMD^8-12BR (Washington, D.C.: January 1988). 
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Findings '^^^ UkeUhood of surviving breast cancer for a specified period of time 

^ depends on a number of factors. These "prognostic indicators" include 

characteristics of the patient (age, race, sex) as well as characteristics of 
the cancer itself (how far it has spread, the types of cells involved, and 
so on). Figure 3.1 shows, by year of diagnosis and for all breast cancer 
patients, the characteristics that both are rented to survival and have 
data available from seer. 



Figure 3.1: Characteristics of All Breast i^^^^^^H 
Cancer Patients in SEER 1975-85 

100 PtrcMtof Cmm 
90 




Ymt of Diagnosis (N u Numbsr of Casss) 

Tumor is In Sftu or confined to breast 
Tumors less than 2 cm at diagnosis 

■■■■ Patients less than 50 years of age 

■■ ■ ■ Nonwhite patients 



As can be seen from this figure, the trend line for each prognostic factor 
is relatively flat. What this means is that we should expect little, if any, 
change in patients* survival to result from a shift in the distribution of 
prognostic factors. This is especially true for patients we selected for 
our study. Figure 3.2 shows trend lines for this group. If anything, the 
trend lines show even less change over time than exhibited by the gen- 
eral population of breast cancer patients. (The line for percent of cases 
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with four or more positive lymph nodes starts in 1977 and ends in 1982 
because data on nodal status were not available for any other years.) 



Figure 3.2: Characteristics of 
Premenopausal, Node-Positive, Stage H 
Breast Cancer Patients in SEER 1S75-85 
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One thing that has changed among the group of patients we selected is 
the frequency with which acUuvant chemotherapy is administered. As 
table 3.1 shows, there was a steady increase in the use of chemotherapy 
from 1975 to 1982." What the table also shows, however, is that 3-, 5-, 
and 7-year survival rates have remained relatively stable across diag- 
nostic cohorts. 



* For reasons that are discussed in our January 1988 report on the use of breakthrough treatments, a 
more precise statement is that there was a decrease in the number of patients not receiving chemo- 
thtrapy. See U.S. General Accounting Office, Cancer Treaqnent 1976-85: The Use of Breakthrough 
Tr eatments for Seven Types of Cancer , GA0/PEMEK88-12BR (Washington, D.C.: January 1988). 

L% Page 20 /^.C GAO/PEMMM The Survival of Breaat Cancer Patienta 

i 



Chapters 

Treatment Patterns and Survival 



Table 3.1: Premenopausal, Node- 
Positive, Stage II Breast Cancer Patients: 
Treatment and Survival Patterns 



Survival rate (.95 confidence Interval)* 





Percent receiving 


3-yea 


r 


5-year 


7-year 


Year of diagnosis 


chemotherapy 


Rate 


CI 


Rate 


CI 


Rate CI 


1975 


23% 


.82 


.036 


.72 


.042 


.64 .045 


1976 


45 


.85 


.033 


.71 


.043 


.64 .045 


1977 


46 


.83 


.036 


.71 


.042 


.64 .045 


1978 


53 


.83 


.035 


.70 


.043 


.63 .045 


1979 


55 


.83 


.035 


.71 


.042 


.65 .044 


1980 


62 


.86 


.031 


.77 


.038 




1981 


66 


.83 


.033 


.72 


.040 




(982 


72 


.84 


.031 








1983 


69 


.85 


.030 









^The survival rates provided assume complete follow-up only through the end of 1985. That is why the 
fast year for 3-year survival is 1983, for 5-year, 1981, and for 7-year, 1979. The figures in parentheses 
define the 95-percent confidence intervals for each survival rate estimate. These intervals are con- 
structed by adding the numbers in parentheses to, and subtracting them from, the estimate provided 
For example, using the data, we conclude that 95 percent of all possible estimates of the 3-year survival 
rate for women diagnosed in 1975 would fall between 78.4 (82 - 3.6) and 85 6 (82 + 3.6) percent 



When confidence intervals are constructed for each of the survival rates 
in table 3.1 (by adding and subtracting the corresponding standard 
error), there is no instance in which the intervals do not overlap. This 
means that there is no statistically significant difference between any of 
the rates. As mentioned in the previous chapter, however, survival rates 
alone can be misleading, since they measure survival only at specific 
times rather than along a continuum. Therefore, we also compared the 
survival of the successive cohorts by using the lifetable method. 

The lifetable method compares actual length of survival (not only per- 
centage surviving at specified times) of all cases across groups and pro- 
vides statistical tests that indicate whether survival is different between 
the groups. In looking for differences in survival, we first compared the 
survival experiences of successive diagnostic cohorts (that is, women 
diagnosed in 1975 with those diagnosed in 1976, those diagnosed in 
1976 with those diagnosed in 1977, and so on, through 1984-85). 
Because of the possibility that there w?s an incremental improvement in 
survival that went unnoticed because we only compared proximate 
cohorts, we also made other comparisons. Among these were 

• a comparison of 1975 (the year in which the smallest percentage of 
patients received a(^juvant chemotherapy) with 1982 (the year in which 
the largest percentage of patients received adjuvant chemotherapy); 
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an analysis that combined 1975, 1976, and 1977 into a single cohort and 
compared that cohort to each of the successive cohorts; and 
an analysis that examined all cohorts simultaneously to see if there was 
any overall significant difference in survival. 

Irrespective of the specific comparisons made, the results were consis- 
tent. Specifically, they show that there has been no detectable change in 
patients' survival since 1975, the year in which ac^uvant chemotherapy 
was proven effective in prolonging the lives of cancer patients. The one 
exception to this finding is that women diagnosed in 1980 did signifi- 
cantly better than women diagnosed the previous year. Unfortunately, 
this increase in survival did not hold steady and the 1981 cohort did 
significantly worse than women whose cancers were discovered in 1980. 
Reinforcing the fmding that the 1980 group is somehow "unique" (and 
we have no description or explanation of this uniqueness) is a compari- 
son that shows no differences between the 1979 and 1981 groups. 

In our final attempt to detect changes in survival, we used proportional 
hazard models. These models show the extent to which patients' charac- 
teristics are related to the probability of dying (relative risk). In our 
study, the characteristic we were most interested in was the year in 
which a patient was diagnosed as having breast cancer. The question 
was whether the probability of death changed from one diagnostic 
cohort to the next. The results of this analysis are displayed in table 3.?. 
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Table 3.2: The Relative Risk of Dying ?or 
Premenopausal, Node-Positive, Stage II 
Breast Cancer Patients by Year of 
Diagnosis 



^Wtnte = 0. nonwhite = 1. 
^Reference year = 1975 



Characteristic 


Beta 


Relative risit 


p-value 


Race^ 


.251 


1.28 


.001 


Age at diagnosis 


-.013 


.99 


.003 


Tumor size 


.358 


1.43 


.000 


Tumor extension 


.258 


1.29 


.000 


Year of diagnosis" 


1976 


.018 


1,02 


.876 


1977 


.045 


1.05 


.693 


1978 


-.020 


.98 


.863 


1979 


-.058 


.94 


.619 


1980 


-.33f. 


.71 


.009 


1981 


.016 


1.02 


.895 


1982 


-.146 


.86 


.259 


1983 


-.126 


.88 


.333 


1984 


.011 


1.01 


.935 


1985 


318 


1.37 


.063 



What table 3.2 shows is that a patient's race and age were both signifi- 
cantly related to the probability of dying (with white patients less likely 
and younger patients more likely to die). The table also shows that the 
extent to which a tumor had spread at the time of diagnosis and its size 
were also related to the probability of death. Finally, with respect to the 
influence of the year of diagnosis, the results of the proportional hazard 
models are entirely consistent with those of the lifetable analyses. That 
is, there is no significant change (p-value less than .05) in the risk of 
dying from one cohort to the next (once again, with the exception of 
1980). 



Conclusions possible? That is, what would explain our consistent finding 

that there is no observable improvement in breast cancer patients' sur- 
vival, even though the use of chemotherapy has increased considerably? 
The most direct, but least likely, explanation is that chemotherapy is not 
beneficial. The procedure through which chemotherapy was shown to 
improve survival is accepted as the best method for demonstrating the 
effectiveness of any therapy. This procedure, known as the randomized 
clinical trial, provides the strongest evidence that any benefits that are 
seen should be considered real. This is especially true when more than 
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one clinical trial shows a therapy to be effective, as with acyuvant chem- 
otherapy for premenopausal breast cancer. Based on the trials con- 
ducted, it has been stated conclusively that chemotherapy should have 
efficacy in the treatment of this group of patients.- 

A number of more likely reasons exist for why patients may not have 
realized the benefits of acyuvant chemotherapy. One factor that clearly 
explains away part of the mystery is that treatment did not change for a 
considerable number of patients. That is, given that approximately one 
in five women already received chemotherapy in 1975, there might be 
little opportunity for improvement among 20 percent of the patients. In 
addivion, it is clear that we should expect no change in survival among 
the patients who never received chemotherapy. Although this number 
diminished over time, approximately a third of the patients in the 1985 
cohort had not received ac^juvant chemotherapy. Among these two 
groups, we would expect little if any change in survival, since patterns 
of treatment remained constant. 

A second reason why patients have not realized the benefits of chemo- 
therapy may be that the chemotherapy they are receiving is inappropri- 
ate. As we discussed in our January 1988 report, seer data are not 
sufficiently precise to inform us as to exactly what therapy v/as used. 
For example, a treatment advance might be the combina.tion of three 
specific drugs into a chemotherapeutic regimen. From seer, however, 
one can tell only whether or not the patient received chemotherapy. The 
SEER data are not detailed enough to indicate the exact type of chemo- 
therapeutic regimen administered. As a result, when we say that there 
has been a twofold-to-threefold increase in the use of chemotherapy for 
breast cancer patients, this does not necessarily mean that all, some, or 
any of these women are being treated with the correct combination of 
drugs, given in proper sequence and at appropriate dosages. That is, the 
unchanging survival experiences of breast cancer patients, year after 
year, could be a function of poor implementation of the breakthrough in 
treatment. 

Finally, although it is acknowledged that ac^juvant chemotherapy is ben- 
eficial in the treatment of premenopausal breast cancer, the question of 
how large a benefit it provides remains unresolved. Different trials have 
reported benefits in terms of both disease-free survival and overall sur- 
vival and have not taken their measurements at a standard time (for 



-National Institutes of Health, Consensus Development Conference Statement, Adjuvant Chemother- 
apyfor Breast Cancer , 6:12 (1985). 
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example, one trial may report only 3-year disease-free survival while 
another may report 9-year overall survival). In addition, even when 
there is consistency in measurement and reporting, the results often dif- 
fer. One consequence of the differences is that it is not clear whether 
our results are surprising or expected. If ac^uvant chemotherapy is 
expected to extend survival for 25 percent of the patients who receive it 
(as HHS stated in its response to an earlier gag report), our finding of no 
detectable ixt^proVement in survival is surprising.^ However, if the true 
benefit of this therapy extends to only 7-to-lO percent of premen- 
opausal, node-positive breast cancer patients (as hhs currently main- 
tains), it is likely that whatever improvements in survival did occur 
after 1975 would be too small to detect with the available data and sta- 
tistical procedures. 



Implications compotlng explanations for why the increased use of 

^ ac^juvant chemotherapy has not led to detectable improvements in 

patients' survival have direct, yet different policy implications. If the 
primary explanation is that many patients are not receiving any adju- 
vant chemotherapy, efforts to increase the use of this therapy are in 
order. These efforts would be directed at patients (if their refusal to 
accept chemotherapy explains their faihire to receive the therapy), at 
physicians (if they do • >t offer chemotherapy to their patients), or at 
both groups. 

If sur\aval rates have not improved because physicians do not provide 
the right kind of chemotherapy, some mechanism must be developed to 
improve the quality of care. More focused training or regulatory efforts 
are but two strategies for achieving this goal. 

However, the third explanation — that the magnitude of the improve- 
ment was too small to be detected with all the available data and statis- 
tical procedures — does not appear to lend itself to any immediate policy 
resolution.^ This is not to suggest that the issue is unimportant, how- 
ever. The size of the benefit provided by ac^uvant chemotherapy is 
clearly of considerable interest to breast cancer patients and their 
physicians. 



•^n 'i. General Accounting Office, Cancer Patient Survival: What Progress Has Been Made > GAG/ 
^'EMD-87-13 (Washington, D.C.: March 1987), pTTK 

^One ; ^^"eiitial resolution would be U. expand the number of cases available for analysis. This option, 
howevei , w .!d require a considerable expansion of the SEER program. 
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The point here is that it is essential to determine wliich of the three 
explanations is correct and that two of these are potentially susceptible 
to resolution. We cannot now recommend a policy to adopt because we 
cannot say which of the three explanations or what combination of 
them more accurately reflects reality. A variety of strategies exist for 
making this determination. Unfortunately, each of these strategies has 
its limitations and none can be assumed to easily provide an answer to 
the question. 

For example, one approach would be to go to patients' case records, 
determine exactly what type of chemotherapy was or was not given, 
and then relate the type of therapy to each patient's survival. The prob- 
lem with this design is twofold: it is expensive, and it depends entirely 
on case records' providing sufficient detail to allow investigators to 
identify the thera^^y that was given. 

An alternative strategy for determining the therapy that was provided 
would be to ask physicians what therapy they gave. This is a less costly 
design than case record review, but it is open to problems of memory as 
well as other forms of response bias (for example, physicians might well 
hesitate to say they did not provide appropriate care). 

The most elaborate (and costly) study design would be one in which 
patients were prospectively followed from the time of diagnosis. By 
observing exactly how the patients were treated and then tracking their 
survival, we would have the strongest evidence on what in fact hap- 
pened and why. Here again, however, there are methodological obsta- 
cles. Aside from the costs and time required to conduct a prospective 
study, there is the issue of physicians' and patients' compliance. If 
either the patients or their physicians were unwilling to cooperate with 
a research team, the conclusions of such a study might be methodologi- 
cally flawed. 

These problems present obstacles that must be overcome if a study is to 
determine conclusively why no improvement in survival is visible. 



We recommend that the secretary of the Department of Health and 
Human Services initiate a study to determine why there has been no 
detectable improvement in the survival of premenopausal, node-positive 
breast cancer patients since the advent of chemotherapy in 1975. In 
light of the potential methodological obstacles that such a study faces, 
we also suggest that prior to conducting it, the secretary seek expert 
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advice on the feasibility of conducting the recommended research and 
the study design most likely to provide valid conclusions. 



The Department of Health and Human Services reviewed a draft of this 
report, hhs concurred with the recommendations contained in that draft, 
although the agency suggested including a feasibility analysis prior to 
the full-scale study. Our current recommendations reflect this 
suggestion. 

HHS summarized its general c^'^^tients by listing three "mgjor concerns'' 
with our report (see appendix III): 

"l.The conclusion, as worded, gives the erroneous impression that no progress 
against Stage 11, premenopausal breast cancer has been made since the advent of 
acljuvant chemotherapy. The body of the report makes it clear that GAO does 
believe there has been improvement as evidenced by clinical trials; however, the 
conclusion as stated gives exactly the opposite impression. 

••2.GA0 interprets its analysis to show that no increase in survival is detectable by 
any means. However, the GAO analysis does not have sufficient statistical power to 
be able to justify this definitive statement. 

"S.The Department agrees with GAO that a closer examination of the actual chemo- 
therapy delivered to breast cancer patients would be useful." 

In reading through the full text of hhs's comments, we concluded that 
the "conclusion** mentioned in the first concern is actually the title of 
the draft report hhs reviewed. We agree that the draft title could have 
been misinterpreted, so we have changed the title to Breast Cancer: 
Patients* Survival . 

The concern that is most pervasive throughout hhs*s comments is 
described in the second point. In its response to our report, hhs consist- 
ently emphasizes the small number of patients included in our study and 
argues that this number leads to our study's having low statistical 
power. Furthermore, hhs contends that low statistical power is the 
mgjor reason that no improvement in survival was detected. That is, the 
small number of patients makes it likely that whatever the improvement 
in survival, improvement would not achieve statistical significance. We 
agree that our findings may result from small sample size. In fact, this 
point was made in the draft hhs reviewed and remains in t^is report. 
However, statistical power is a function of several factors: in addition to 
sample size, the most important factor in our study is the size of change 



Agency Comments 
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in survival one expects (effect size). As either sample size or the size of 
the expected change decreases, so does the statistical power. 

In our analyses, the size of the samples is a given: we included all breast 
cancer patients from SE£}< who satisfied NCi's criteria for defining ideal 
candidates for ac^uvant chemotherapy. What is less clear is how large a 
change in patients' survival should be expected. In its response to a 
1987 GAG report, khs stated that a 25-percent improvement in survival 
was to be expected for premenopausal breast cancer patients.'^ If this 25- 
percent estimate of effect size is used, the power of our statistical tests 
is adequate. However, if hhs's current estimate of 7-to- 10-percent is 
closer to the true effect size, our statistical tests do not have a reason- 
able probability of detecting that effect. A question remains, however, 
as to the basis for the change in hhs's estimate of the likely survival 
benefits from ac^uvant chemotherapy. 

In that earlier response, hhs also argued that survival rates could be 
improved significantly **through better application of existing treat- 
ments." This position of hhs's supports our conclusion that poor imple- 
mentation may explain why we did not detect any improvement in 
patients' survival. 

There are, therefore, a number of possible explanations for our findings: 
the number of patients available for our study was small; the benefits of 
ac^uvant chemotherapy are small; many physicians do not give chemo- 
therapy correctly, hhs, in its responses to this and earlier gag reports, 
has agreed with all these factors. 

All other comments provided by hhs have been considered and changes 
have been made in the report as appropriate. 



^*U.S. General Accounting Office, Cancor Patient Survival: What Progress Has Been Made? GAG/ 
PEMD-87-13 (Washington, D.C.: March :987), p. 1 14. 
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The classic problem in survival analysis is that of censoring or right 
truncation . At issue is what values to assign to the survival times of 
patients who are still alive at the conclusion of a study. If we simply 
calculate their survival by time at end minus time of entry, we are 
equating their survival to that of patients who entered the study at the 
same time and died near the time that the study ended. 

As a solution to the censoring problem, many algorithms add time to 
anyone who is censored (alive when last seen). Typically, the amount 
added is half the time in the subsequent interval. For example, any 
patient who was alive at the end of our study would be given 6 addi- 
tional months of survival, if we measured survival in years. This 
method of assigning values to censored cases introduces a bias, because 
of the natural history of breast cancer and the types of patients we 
included in our analyses. 

Our study was limited to Stage II, premenopausal patients who had had 
surgery. Among this group of women, it is very unusual for anyone to 
die (from cancer or other causes) within the first year or two of diagno- 
sis. This means that assigning half an interval to censored data is likely 
to bias later diagnostic cohorts. For example, consider a patient diag- 
nosed in January 1975 and another patient diagnosed in January 1987. 
Assume that the probability of death during the first 2 years following 
diagnosis is small for both patients. Finally, remember that our study 
ends in January 1988 with the second patient still alive. 

Since the study ends with the second patient still alive, her survival time 
has to be estimated. This estimate measures her survival as 18 months 
(the 1 year she was measured while alive and the additional half year 
she gets for being censored). However, the likelihood of her surviving to 
the end of the second year is great. This means that 24 months (at a 
minimum) is probably a better estimate. 

The probleut was aggravated by the fact that our initial censoring date 
was relatively early (December 31, 1985). We believe that the early cen- 
soring date and the bias described above accounted for initial results 
that showed that women in our last two cohorts had significantly worse 
survival than other patients. That is, women in 1975, 1976, 1977, and so 
on had the opportunity to live 2 and 3 years (which most did) whereas 
those diagnosed in 1985 did not. 

To correct for this problem, we extended the censoring date to December 
31, 1986. Since we did not have complete follow-up on patients up to 
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that date, we assumed that any patient who was not recorded as dead 
was alive at the end date. When this was done, the differences between 
the 1984 and 1985 cohorts relative to all others disappeared. 



Estimating Cohort models, the year of diagnosis was entered into the model in 

Vffckof different ways. One was to record the direct value (for example, 

mieCl 1977, 1978, 1979) of the year in which the patient was diagnosed. We 

also constructed 1 1 dummy variables to correspond to each of the 1 1 
annual diagnostic cohorts. It is this latter measurement scheme that is 
reported in chapter 3. 

Also at issue was which cohort should be used as the comparison group 
for the proportional hazard models. We chose 1975 for the theoretical 
reason that it was the year in which the advance in treatment was first 
reported. However, when we combined 1975 and 1976 or combined 
these two cohorts with 1978 to fonr. a comparison cohort, the results 
remained unchanged (1980 is the only significantly different cohort). 



StSltiStiCal Power of conducting climcal trials, it is advisable to determine the statistical 

fha <?tiirlTr Flacic^n ^^^^ ^^^^^^ ^^^^ probability that an effect will be detected if one 

Lne olUay UeSlgll really exists) as ea^ly as possible. Among the primary benefits of power 

estimates is that they allow for modifications in the study design (for 
example, expanding the number of patients to be enrolled) while such 
modifications are still easily made. In fact, the utility of power estimates 
for determining sample size is so well established that tables exist that 
show the required number of subjects for each desired level of signifi- 
cance and statistical power.' 



In situations where the analyst is unable to modify the study environ- 
ment (by increasing either the effect size or the sample size), power esti- 
mates serve more to help in the interpretation of study findings than to 
inform design decisions. The study described by this report is one in 
which both the effect size and the number of patients included in the 
analyses were factors beyond the control of the analysts. The utility of 
any power estimates for this study, therefore, derives primarily from 
their role in helping in the interpretation of our findings. 



or example, L.S. Freedman, "Tables of the Number of Patients Required in Clinical Tnals Using the 
Txgrank Test/' Statistics in Medicine , 1 (1982), 121-29. 
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The mgjor finding of our study is that there was no detectable improve- 
ment in patient survival over time. If the power of our study is high, thL' 
finding would be more accurately stated by onutting the word **detecta- 
ble" from it. If the power of our study is low, the finding is correctly 
stated as is and should come as no surprise. (That is, we could not detect 
any change primarily because the change is difficult to detect with the 
design we employed.) 

Obviously, the alternative conclusions to be drawn from our study, 
given high or low power, have different implications. A conclusion that 
there has been no improvement in survival could point to a problem in 
the way cancer patients are treated. A failure to detect improvement 
could mean no more than the obvious fact that small effects are difficult 
to detect. Unfortunately, we could not determine which of these per- 
spectives is more accurate because we could not compute the power of 
our design. 

One reason we could not estimsts the power of our study is that there is 
no agreed-upon estimate of the magratude of the benefit provided by 
acyuvant chemotherapy. As we indicate in chapter 3, hhs has itself pro- 
vided two distinct estimates of effect fiize, each of which would lead to 
considerably different estimates of power. 

A second reason that we could not estimate power is that the central 
analyses (the lifeiable and proportional hazard comparisons) were per- 
formed by making numerous comparisons across the 11 cohorts rather 
than a simple comparison of one cohort against a second. This presents a 
number of problems for power computations. One obvious (and insur- 
mountable) one is that we could find no existing algorithm for comput- 
ing power for an n-way logrank test. An equally vexing dilemma was 
what comparisons should be included: 1975 against all other years, 1975 
and 1976 against all other years, 1975 through 1977 against 1982 
through 1985, and so on. Since each of these tests was run, and each 
would generate different power estimates as a result of different sample 
sizes, we could generate a wide range of power estimates. 
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Patient Selection Criteria 



In an earlier study on the use of state-of-the-art cancer therapies, we 
asked nci to provide us with criteria for selecting the cancer patients 
who should be the most likely candidates for adjuvant chemotherapy.' 
These criteria were applied to the entire population of breast cancer 
patients contained in seer. The patierts who satisfied all the criteria 
were included in the earlier study and also served as the population for 
the current study. 



The sex of a patient had to be female. 

The age of a patient had to be 50 years or less at the time of diagnosis. 

The medical history of a patient could not include any previous diagno- 
sis of cancer. 

The diagnosis must have been made through some means other than an 
autopsy. 

The diagnosis must have been made in 1975 or later . 
The patient must have had surgery . 

Tumor Characteristics '^^^ primary site of the tumor was in the breast. 

The size of the tumor at the time of diagnosis was less than fy.O 
centimeters. 

The histology of the tumor was coded as 8140 (adenocarcinoma, Not 
Otherwise Specified), 8141 (scirrhous adenocarcinoma), or 8500 (infil- 
trating duct carcinoma), using the International Classification of Dis- 
eases , 9th Edition, Oncology. 

The tumor could not extend to or beyond the chest wall, and there could 
be no evidence of metastatic activity to adjacent or distant organs. 

The nodal status of the patient had to be positive with only the axillary 
lymph nodes involved and no involvement of distant lymph nodes. 



'U.S. General Accounting Office, Cancer Treafanent 1975-85; The Use of Breakthrough Treatments 
for Seven Types of Cancer , GAO/F12MU-88-12BR (Washington. D.C.: January l&gd), pp. 1^. 
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f Comments From the Department of Health and 
' Human Services 




DEPARTMENT OF HEALTH & HUMAN SERVICES 



Office of inspector General 



Washington. OC 20201 



OCT I 8 1988 



Ms. Eleanor Chelimsky 

Di rector / Program Evaluat ion 

and Methodology Division 
U.S. General Accounting Office 
Washington , D.C. 201'* 8 



Enclosed are the Department's comments on your draft report/ 
"Breast Cancer: Patients Have Yet to Realize the Benefits of 
Adjuvant Chemotherapy." The enclosed comments represent the 
tentative position of the Department and are subject to 
reevaluat ion when the final version of this report is received. 

The Department appreciates the opportunity to comment on this 
draft report before its publication. 




Sincerely yours/ 




Richard P. Kusserow 
Inspector General 



Enclosure 
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C OhMENTS QF THE DEPftRTMENT OH HEftLT H^ ftND HUMftN SERVICES 
ON THE GENERAL ACCOUNTING OFFICE'S DRftFT REPO RT. 
"B REftST CA N CER! P A TIENTS HftVE YET TO REALIZE THE BENEFITS 
OF ADJUVANT CHEM OTHERAPY ,J1 SEPTEMBER 1<988 



General Camments 

The Department appreciates the opportunity to comment or\ the 
General Accounting 0-f-fice (GAO) drart report. In its report, the 
GAO attempts to determine whether an increase in the use of 

djuvant chemotherapy has led to a population-wide improvement 
in survival -for women with Stage II breast cancer, under the age 
o-f 50, since 1975. 

GAO concludes "that despite a considerable increase m the use of 
chemotherapy since 1975, there has been no detectable increase in 
survival -for those patients who should have benefited most -from 
the advent o-f this therapy." GAO recommends that the Secretary 
o-f the Department o-f Health and Human Services (DHH5) initiate a 
study to determine "why there has been no visible improvement in 
the survival o-f premenopausal, node-posi ti ve , breast cancer 
patients despite the advent o-f adjuvant chemotherapy." 

The Depa-tment agrees with the conclusion drawn in the GAO report 
that adjuvant chemotherapy clearly prolongs the survival -for 
premenopausal women with Stage II breast cancer. As pointed out 
by GAO, this survival advantage has been well documented in 
several, large, randomized clinical studies, and the Department 
agrees with GAO that "the question o-f whether (adjuvant) 
chemotherapy has the potential to e:;tend patient survival has 
bee*-, settled." 

The Department also shares GAO ' s concern that this survival 
advantage may not be reaching the entire population o-f Stage II, 
premenopausal breast cancer patients nationally. Our concerns 
are two--fold; -first, that not all eligible patients are receiving 
adjuvant chemotherapy, a concern borne out by the GAO analysis 
which shows that 31 percent o-f eligible patients did not receive 
chemotherapy as late as 1983; and second, that patients who are 
being treated may not be receiving chemotherapy with the 
intensity (dosage and timing o-f treatment) needed to achieve the 
potential survival advantage. 

The Departoient believes, however, that the statistical power of 
the GAO analysis is not sufficiently strong to allow the sweeping 
conclusion that no increase in survival benefit can be detected. 
The major reason -for this is that the Surveillance, Epidemiology 
anci End Resu.ts (SEER) database contains too -few Stage II, 
premenopausal patients who meet the GAO selection criteria 
(approximately 400 to 650 per year), to be able to draw a 
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definitive conclusion given the magnitude oi the survival 
advantage expected based on clinical trials data (7 to 10 
percentage points at 5 years post diagnosis) and given the 
statistical approach used by GAO. The National Cancer Institute 
IS supporting extensive research to develop therapies which will 
confer greater survival advantages. However, while this 7 to 10 
percent survival improvement represents a si'jnif icant 
accomplishment of adjuvant chemotherapy^ de<:ecting this 
difference using the methods employed by GAO would require two to 
three times as many patients as are avai lable in the SEER 
database. The Department's analysis indicates that the GAO 
approach had less than a 50 percent chance of demonstrating an 
improvement in 5 year survival using the SEER database. This 
a.eans that there was at least a 50 percent chance that the GAO 
analysis would miss finding a survival advantage even if one 
existed . 

The Department believes that an incomplete transfer of the 
<*djuvant chemotherapy treatment advance to the community also 
contributed to GAO not detecting a survival advantage between 
1975 and 1983. The fact that only 69 percent of eligible 
patients received adjuvant chemotherapy in 1983 speaks to this 
point. On a recent analysis of the SEER database, the Department 
also found that there were proportionately more patients wi th 
four or more positive lymph nodes in the treatment group than in 
the overall SEf-R population of Stage II, premenopausal breast 
cancer patients. This wc*,ild resul t in a smal ler than expected 
survival benef i t for the treatment group since patients wi th four 
or more positive lymph noc'e«i receive a lesser benefit from 
adjuvant chemotherapy than does the general population o f 
premenopausal, Stage II breast cancer patients (Bonadonna G., 
Rossi A., Tancini G. et al. (1983) "Adjuvant Chemotherapy in 
Breast Cancer" (letter) LANCET, 1, 1157). The Department 
believes that this incomplete transfer of adjuvant chemotherapy 
to the community may have been a major contributing factor to a 
lower than ex pec ted national survival benefit between 1975 ^nd 
1983. 

Although the SEER database is the best currently existing 
resource to have u£>ed for the GAO study* it does not contain 
enough information about patient treatment to definitively answer 
questions about the impact of particular treatments on survival 
and about patterns of care. The SEER database does not capture 
information about the nature of treatment (single agent versus 
combination chemotherapy), the dosages given, or the length of 
treatment . It is , therefore , not possible to determine whether 
adjuvant therapy is being given in the community using the same 
methods which improve surv : val in c 1 inical trials . 
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The main conclusion oi the GAO dra-ft report is stated in the 
title, "Breast Cancer: Patients Have Yet to Realize the Benefits 

0- f Adjuvant Chemotherapy". The Department believes that this 
statement is unin-f ormative and possibly misleading. I -f this 
conclusion means that the breast cancer patients who actually 
received adjuvant chemotherapy showed no survival bene -fit, we 
disagree; GAO did not provide evidence to support this assertion. 

1- f it means that there has been no possibly discernible impact on 
the whole Stage II, premenopausal population,, we also disagree. 
As stated above, the GAO analysis does not have su-f-ficient 
statistical power to prove that a survival bene-fit -from adjuvant 
chemotherapy has not occurred national ly . The Department 
suggests that a more appropriate title -for the GAO report might 
be, "Adjuvant Chemotherapy -for Breast Cancer; Is a Survival 
Benp-fit Detectable in the National Statistics*^" The use o-f this 
alternative title could prevent a misunderstanding about the -fact 
that adj uvan t therapy is an e-f -f ec tive method o-f treatment . 

In cone 1 us ion , the Department has three major concerns with the 
GAO report; 

1. The conclusion, as worded, gives the erroneous impression 
that no progress against Stage II, premenopausal breast 
cancer has been made since the advent o-f ad j uvan t 
chemotherapy. The body o-f the report ma*;es it clear that 
GAO does believe there has been improvement as evidenced by 
clinr.cal trials; however, the conclusion as stated gives 
exactly the opposi te impression . 

2. GAO interprets its analysis to show that no increase in 
survival is detectable by any means. However, the GAO 
analysis does not have su-f-ficient statistical power to be 
able to justi-fy this de-finitive statement. 

3. The Department agrees wi th GAO that a c loser e;;ami nation o-f 
the actual chemotherapy delivered to breast cancer patients 
would be use-f ul . 

GAO Recommendation 

GAO recommends that the Secretary o-f the Department o-f Health and 
Human Services <HHS> initiate a study to determine why there has 
been no detec table improvement in th'. survival o-f premenopausal , 
node-positive, breast cancer patients since the advent o-f 
adjuvant chemotherapy in 1975. GAO also recommends that prior to 
initiiition o-f such a study, the Secretary convene and seek the 
advice o-f an advisory committee as to the design most likely to 
provide val id cone 1 us ions. This commi ttee should inc lude 
representatives -from the cancer research community, practicing 
oncologists, physicians other than oncologists who have a role in 
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cancer patient management and women who have been treated -for 
breast cancer. Care should be exercised in selecting the 
participants to ensure that the major pro-f essiona 1 societies are 
also represented. This is to increase the likelihood o-f 
cooperation by all relevant parties with any study endorsed by 
this group. 

Department Response 

The Department agrees with the GAO recommendation, but not with 
the way it is stated. As mentioned in the General Comments 
section, and described in more detail in the Technical Comments 
section, the GAD analysis does not have the statistical power to 
state that there has been no improvement in survival -for Stage 
II, premenopausal breast cancer patients. The number o-f women 
in the SEER database, in the category selected by GAO, is too 
small (400 to 650 per year) to be able to detect a change in 
survival o-f the size predicted -from the clinical trials 
literature using the GAO approach. It is, there-fore, not 
appropriate -for GAO to recommend a study to determine why there 
has been no apparent survival improvement, but rather whether 
adjuvant therapy -for breast cancer has been successfully 
trans-ferred -from clinical trials to clinical practice. 

A patterns o-f care study o-f the delivery o-f adjuvant therapy -for 
breast cancer, not just in Stage II premenopausal women, would be 
important and could provide valuable insights to aid the transfer 
o-f clinical trials advances to clinical practice. The Departmen.: 
agrees that an advisory panel s*iould be convened prior to the 
initiation o-f a study, not only to advise on study design, but to 
assess the -feasibility o-f success-f ul ly conducting such a study. 
The availability and adequacy o-f patient records are the 'ajor 
•factors which in-fluence the -feasibility o-f a patterns o-f care 
study. Any study design chosen will require the complete 
availability o-f patient records to provide detailed in-formation 
about chemotherapy agents used, dosage, -frequency, and length o-f 
treatment. In a clinical trial, such data collection and 
monitoring is standard and agreed upon, in advance, by the 
patient and physic la.i. For a patterns o-f care study o-f the 
delivery o-f therapy which is principally done in indiviiual 
physicians' o-f-fices, physicians would have to agree to ^eep 
records in greater detail than is customary. Furthermore, access 
to records o-f patients not participating in a clinical trial 
would be essential and would require an unprecedented level o-f 
cooperation and openness on the part o-f physicians and their 
patients. Should the advisory committee determine that a 
patterns o-f care study is -feasible, it would be advisable to 
conduct a pilot study prior to the -full scale e-^fort to be 
certain that the necessary in-formation can be obtained. 
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Additionally, a patterns o-f care study would require a 
signi-ficant commitment o-f personnel, equipment, and -financial 
resources over a several year period. 



Technical Comments 



Title Page 



The title o-f the report. Breast Cancer; Patients Have Yet to 
Realize the Bene-fits o-f Adj i' ^ ant Chemotherapy asserts a 
conclusion that cannot be su^^ported by the report itsel-f, does 
not accurately re-f 1 ec t the contents , and could suggest to the 
public that adjuvant chemotherapy is not an e-f-fective method o-f 
treatment. GAD speci-f ical ly states that it does not attack the 
positive -findings -for adjuvant chemotherapy in clinical trials, 
but the title clearly states patients have yet to realize the 
bene-fits o-f adjuvant chemotherapy. 6A0 -failed to -find a 
survival improvement re-f lec ted m national statistics -for Stage 
II, premenopausal patients, but the size o-f the sample and the 
treatment e-f-fect which could be e^;pected based on a care-ful 
analysis o-f the clinical trials oata made the lil-elihood o-f 
-finding this improvement less than 50 percent. 

The report notes that 31 percent o-f the premenopausal women who 
have Stage II breast cancer, a stage -for which chemotherapy is 
indicated, did not receive adjuvant therapy in 1983. The title 
o-f the report could inadvertently lead to a decrease in the 
percentage o-f women who receive adjuvant chemotherapy as it 
leaves the mistat en impression that this treatment, which has 
well ^ nown side e-f-fects, is useless. 



Now page 2. 



Now page 3 
• Now pages 2 and 3. 



Page ES-1. Paragraph 1 

The report opens with the assertion that the Nation spenos 
billions o-f dollars on new medical technologies. Since the 
passage o-f the National Cancer Act in 1971, the National Cancer 
Institute has spent a total o^ *4.55 billion on all forms ot 
cancer treatment research out o-f its total appropriation o-f $14. 
billion during this time period. The largest expenditures have 
been -for basic research and have led to an unp 'eceden ted 
understanding o-f the nature of cancer. 

Page ES-4 . Table 1 

GAD'S multivariate analyses on pages r-B include cases through 
1985, yet Table 1 stops at 1<983. li 1985 cases are part ct the 
GAO analysis, they should also be included in "Table 1. 
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Now page 3. 



Now pages 7 and 9. 



Now page 8. 



Page ES~5. Paragraph 1 

6A0 describes several plausible reasons that might contribute to 
the -failure to detect benefits of adjuvant chemotherapy in 
national survival statistics. It omits the -fact that many 
physicians who are hesitant to prescribe chemotherapy do so only 
-for t^'^ir patients with the worst orognoses; large primary tumor, 
more *:h«fc, -four positive lymph nodes, aggressive histology, etc. 
The Department's analysis of the SEER data supoorts this 
assprtioi an. shows that there were proportionately more patients 
with four or more positive lymph nodes in the treatment group 
than in the whole SEER Stage II, premenopausal population. This 
would result in a smaller survival benefit for the treatment 
group since patients with four or more positive lymph nodes 
receive a lesser benefit from adj uvant chemotherapy than does the 
overall population of premenopausal, node-positive breast cancer 
patients (Bonadonna 6., Rossi A., Tancini 6. et al. (1983) 
"Adjuvant Chemotherapy in Breast Cancer" (letter) LANCET, 1, 
1157) . 

In addition to these factors, 6A0 should note that the small 
number of breast cancer patients in the SEER database who fit the 
criteria for its study (428 to 653 each year), and the modest 
size of the expected tj-in (7.3 to 10.8 Percentage points in 
clinical trials) resul t in a se'^ious 1 ack of power in the 
statistical amlyses. This makes it highly improbable that a 
national benefit could be detected «*ven if the women who were 
given a"* uvant chemotherapy did "lave their lives extended. It 
should o« noted that clinical trials can op tec t this survival 
difference enrol ling a smal ler number of patients because a 
survival difference is being sought between two populations, 
100 percent of whom r jceive treatment on one arm of the study and 
zero percent who are treated on the other arm. In the SEEK data 
analyzed by GAO, the comparison is made between the 23 to 71 
pe lucent who received treatment over the time period of the study , 
a change in treatment status of only 48 percent. This makes 
larger numbers of patients necessary to detect an evolving 
surviva 1 a in t age . 

Page TC~3. Abbreviations and Page 1-3 

L~PAM IS an acronym for L-phenylalanine mustard, not for 
L-pheny lalanine (which is a non~cy totoxic amino acid) . 



Page j-1 



the 



Reference is made to the focus of the GAO report being " 
advance made when it was discovered ^hat chemotherapy 
administerr d fol lowing surrjery ( ad j uvant chemotherapy ) improves 
the survival chances of premenopausal breast CAncer patien -s. " 
The group referred to in the* report is actually women with 
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ivow page 8. 



Now page 9. 



Deleted. 



Deleted. 



Page 7 



evidence o-f spread the breast cancer into axillary (underarm) 
lymph nodes (node-positive), not all premenopausal breast cancer 
patients. In addition, the term adjuvant chemotherapy should be 
more clearly defined. It refers only to chemotherapy given to 
women with no detectable disease -following definitive local 
therapy (surgery or radiotherapy) in an attempt to eradicatt^ 
undetectable, microscopic cancer cells which may remain in the 
body and does not include chemotherapy given to patients who have 
evidence o-f metastatic disease. 

Page 1-2 

The chronology oi cancer drug development is inaccurate. The 
"first use of antitumor agents in humans was in 1943 when 
alkylating agents were first used at Yale to treat patients with 
leukemia and Hodgkin's disease. The first curative chemotherapy 
was reported in the late 1950' s when methotrexate was used in the 
treatment of metastatic choriocarcinoma. Curative combination 
chemotherapy of many of the leukemias and lymphomas was developed 
in the I960' s. 

Page 1-3 

The GAO report states that "... drugs showed little promise in 
treating carcinomas ..." until the mid-1970's. *»'his is 
incorrect. By the mid-1970's, chemotherapy had oeen shown to 
have significant antitumor activity in choriocarcinoma, breast 
CA-icer, ovarian carcinoma, and small cell carcinome* pf the lung. 
The impact of effective chemotherapy for these carcinomas is 
further magnified when the young median age of patients with some 
of these cancers is taf en into account. Although more effective 
cheno therapeutic reg imens have since been d eve loped for these 
carcinomas, the contributions of these early regimens in terms of 
tumor response and overall survival should not be underes tif.iated . 

Page t-4 

Clinical trials of adjuvant chemotherapy in premenopausal, node- 
positive breast cancer patients have consistently shown a 30 to 
50 percent reduction in relapse rate at 5 years. The 75 percent 
reduction referred to in the GAO report was reported in only one 
study and is not considered a standard estimate of the 
effectiveness of adjuvant chemotherapy in this population. 
Further, this reduction in relapse rate does not translate to an 
equivalent change in the probability of survival. 

Page 1-6, Ref. 12 

NA* \ when used in this context, stands for Nolvade>* (the trade 
name for tamoxifen) Adjuvant Trial Organisation. 
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Pc^ge 8 



Now page 12. 



P aq»? 2~2, Paragraph 



Now page 24. 



Now page 12. 

Now page 30. 
Deleted. 



The wording o-f the -fourth 1 ine could be construed to mean that 
SEER contacts all patients directly -for -follow-up. The SEER data 
v>ase IS deve 1 oped -from hospital records , death records , and 
sometimes 1 etters to the patien t * s physic lan m order to obtain 
the most recent -fol low-up in-formation . However , SEER registries 
almost never contact patients directly. 

Contrary to the statement in the report, the SEER database does 
not contain in-formation on "... how the patient was treated . " 
SEER data does not contain in-formation on the nature o-f the 
treatmen t (i.e. sing 1 e agen t or combination chemotherapy ) , the 
duration o-f the therapy (i.e. single course versus six or more 
eye les o-f therapy ) , and the dose- in tensity (i.e. amoun t o-f drug 
given over a unit period o-f time), all o-f which have been shown 
to sign a -f ican tly m-f luence the outcome o-f patients treated with 
adjuvant chemotherapy. SEER does not include m-formation on 
whether patients receive therapy equivalent to that shown to be 
e-f-fective m cl inical trials. These 1 imitations o-f SEER are 
described only near the end o-f the report, on pages 3-12 and 3- 
13, but should be mentioned in the Objectives, Scope, and 
Methodology portion o-f '-iie report as well. 

GAD reports that SEER provides in-formation on patient 

characteristics (age, race and se>; ) that are use-ful -for c eating 
homogeneous strata o-f patient5 -for analysis. " GftC is correct 
that SEER reports the 1 j sted charac teristics . However , a number 
o-f -factors which could si'^ni-f icantly in-fluence response to 
adjuvant chemotherapy are not reported in SEER, including 
important variables such as estrogen receptor status. In 
assessing a population's response to adjuvant chemotherapy, it is 
importan t that pr ognos tic -features be ^ nown in order to be able 
to predict the magnitude o-f the expected survival bene-fit. 

Page 2-2 « Paragraph 3 

The text should ma^e clear that 3--year -follow-up data was not 
available -for the patients entereo in + o the GAO study who were 
diagnosed in 1984 and 1985. The problems created by the 
censoring methods used to deal with this are discussed m the 
Technical Comments on Page I-l . 

Page 2-5. Paragraph 1 

GAO states that "This disease is diagnosed in appro;' imately 
120,000 women . . . the second most prevalent -form o-f cancer. . . " 
This statement -follows one which limits the di^icussior* to only 
premenopausal women. The 120,000 represents all new cases o-f 
breast cancer diagnosed in 1985, rot just the premenopausal 
subset which constitutes approximately 22 percerit oi the total . 
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Breast cancer has the second highest incidence rate after lung 
cancer. However, this does not imply that it is the "second most 
prevalent" -Form o-F cancer. Preva l ence , ur cumulative incidence 
up to the patient s present age, means the number of people 
currently alive who have a history o-F a particular cancer. 
Because o-F the much longer survival -For breast cancer than -For 
lung cancer, there are many more persons alive today with a 
history o-F breast cancer than o-F lung cancer (Feldman A.R. , 
Kessler L. , Myers M., and Naughton M.D. (1986) "The Prevalence o-F 
Cancer: Estimates Based on the Connecticut Tumor Registry" NEW 
ENGLAND JOURNAL OP MEDICINE, 315, 1394-1397). 

P age 2-5. Paranraoh 1 to 2-6, Paraqrajh 1 

GAO should succinctly state the selection criteria -For these 
analyses. This in-Formation , as well as the number o-F women per 
year included in the group studied, should be included in the 
text. Pigure 3.2 shows that the number o-F women in the study 
cohort -For 1975 was 429, rising in 1983 to 653, with intermediate 
size cohorts in the intervening years. The small number o-F cases 
on which this analysis is based should be clearly stated in the 
report, not just in a -Figure. 

The report says that "cancer p. ' '.ents under the age of 50" were 
included in the study grcup. However, the SEER data provided by 
the National Cancsr Institute at GAO's request, included -Females 
aged 50 and under, a somewhat larger group. The above sentence 
should be revised to say, "patients 50 and under" i-F this was the 
populatior. o-F women studied. The description o-F the study cohort 
should include the histologies o-F the breast cancers which were 
included -For analysis. This in-Formation was omitted -From the 
description o-f the methodology. Page I-l nontions that only 
surgically treated patients were included, but this -Fact is not 
stated in T he Selection o-F Patients and it is not made clear 
whether that selection was indeed made. It would aid the 
interpretation o-F this study i-F the selection criteria were 
presented in tabular -Form indica ing how many cases each 
criterion included as compared to the total SEER node positive, 
premenopausal breast cancer population. 

Page 2-n 

GAO oversti Les the power o-F the statistics which they nmploy in 
their an-r. 1 'SIS in concluding "I-F the coe-F-Ficients did not 
achieve signi-Ficance, we concluded that any observed change was 
probably due to chance." I-F the tests achieve signi-Ficdnce, then 
the e-F-Fect hat» a low probability (usually 5 percent) o-F being a 
spurious -Finding. However, the opposite is not true. There can 
be many reasons, why, when an e-F-Fect is real, the tests do not 
achieve statistical signi-Ficance . Some o-F the reasons include an 
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Page 10 



insu-f -f icient number o-f cases, the proportion o-f cases a-f-fectecJ is> 
small, the e-f-fect being measured \s small, the power is wea^ , 
and/or the model is inappropriate. 



Now page 20. 



Page 3-5 

Although SEER shows that there h^s been a steadv increase in the 
use o-f chemotherapy -from 1975 to 1982, SEER dc ^ not include 
information on the number o-f cycles o-f chemotherapy, the 
combination o-f agents used or the dosage, all -factors which have 
been shown in clinical investigations to -*xert a major in-fluence 
on the e-f -f ectiveness o-f the treatment. 



Now page 22. 



Now page 23. 



Page 3-7 

The statement is made here and elsewhere "... there has been no 
detectable change in pc.tient survival since 1975..." The 
Di?par tment bel ieves that this wording is mislefding . I -f GAO 
wishes to summarize the results o-f its analysis more accurately, 
the statement should read, "In our analysis o-f the SEER data, 
GAO was unable to detect a signi-ficant improvement in survival iii 
this group o-f breast cancer patients." Similar changes should be 
made on Page 3-9, Paragraph 2 and Page 3-10, Line 1. 



The -first step in conducting a study such as that undertaken by 
GAO would be to determine, given the available data, the 
probability o-f beinq able to detect a survival di-f-ference between 
groups i-f one e xisted. This concept is termed the power o-f the 
study. To calculate power^ an estimate o-f survival, assuming 
maximal bene-fit o-f ijuvant chemotherapy, must be provided. 
National Cancer Insti tute scientists using the same in -format ion 
as that available to GAO have made the -following calculations. 
BasecJ on review o-f the papers by Moon et al. (Jones S.E. , Moon 
T.E.,' Bonadonna G., Valagussa P., Rivkin S. , Buzdar A., Montague, 
E., and Powles T. (1987) "Comparison o-f Di-f-ferent Trials o-f 
Adjuvant Chemotherapy in St^gf? II Breast Cancer Using a Natural 
History Data Base" AMERICAN JOURNAL Of" CLINICAL ONCOLOGY, 10(5) 
387-s>:95; and Moon T.E., Jones S.E., Bonadonna G., Valagussa P., 
Powles T., Buzdar A., and Montague E. (1987) "Development and 
Use o-f a Natural History Data Base o-f Bre-^^st Cancer Studies" 
AMERICAN JOURNAL OF CLINICAL ONCOLOGY, 10(5) 396-403), the 
Department estimates that the 5 year survival -for w^men on 
clinical trials under 50 who have positive nodes and tumor sxze 
less than 5 centime»ters is 75.5 percent -for those receiving 
adjuvant chemotherapy compared to 64.7 percent -for those who Jo 
not. For a particular diagnor.AS-year : the SEER data only a 
percentage o-f the patients had adjuvant chemotherapy. Thus, for 
any given year we can calculate the expected 5-year survival as: 
( .755) (percent receiving adjuvant chemotherapy) + ( .647) 
(percent not receiving adjuvant chemotherapy). 
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In the SEER database, there are several chemotherapy usage 

categories: 1) no chemotherapy; 2) chemotherapy received; 

3 ) chemotherapy recommended - unknown i-f received ; 4 ) unt nown t-f 

chemotherapy received . I -f we assume that everyone who had 

chemotherapy recommended < group 3 ) received 1 1 , then Table 1 

shows the expected 5 year survival -for each diagnosis year -from 

1975-1985. 

Table 1 











5 YR 


DX YEAR 


7. RECEIVED 


i CHEMO RECOMMENDED, 


TOTAL 7. 


EXPECTED 




CHEMO 


UNK. IF RECFWED 


CHEMO 


SURVIVAL 


1975 


19.6 


3-3 


o 

. 7 


.672 


1976 


33.8 


1 1.6 


45-4 


.696 


1977 


34.0 


1 1.9 


45.9 


.697 


1978 


45. T 


6.8 


52.7 


. 704 


1979 


45.2 


9.8 


55.0 


. 706 


1980 


48.8 


12.5 


61-3 


.713 


1981 


54.0 


12.1 


66. 1 


.718 


1982 


55.5 


15.9 


71 .4 


.724 


1983 


54. 1 


14.6 


68.7 


.721 


1984 


45.7 


21.0 


66.7 


.719 


19£- 


47.9 


15.0 


62.9 


.715 


Thus , a 


maximum di-f-ference o-f 5.2 percentage 


points in 


5 year 


survival 


( 1975 to 1982) 


would be expected based on the 


survival 


bene-f 1 1 


o-f adjuvant chemotherapy demonstrated 


in clinic 


al trials 



Power ccilcul ations were per-formed using the results o-f Freedman 
(Freedman L.S. (1982) "Tables o-f the Number o-f Patients Requi-^ed 
in Clinical Trials Using the Logrant. Test" STATISTICS IN 
MEDICINE, 1, 121-129) -for comparisons o-f two survival cur'ves 
with the logranl^ test. Figures 1 and 2 show power versus number 
o-f events (deaths) needed (in both groups together) to detect a 
change in 5 year survival -from .672 to .71 and .72, respectively. 
(Note the: the x-axis in Figures 1 and 2 are on di-f-ferent 
scales.) In both curves we assume a one sided Type I error rate 
o-f .05. I-f we assume that there are about 300 events available 
-for analysis (which is approximately the number o-f deaths 
available -from SEER for each pair o-f years being compared) then 
the power to detect a change -from .672 to .71 is between 35 and 
40 percent, and the power to detect a change -from -672 to .72 is 
only 50 percent. This level o-f power is unacceptable because the 
chance o-f missing the expected mprovement in survival due to 
chemotherapy is 50 percent or more- 
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FIGURE 1 

POWER TO CSSTECT A DIFFERENCE IN FIVE YEAR SURVIVAL FPCtl .67 TO ,71 
(HAZARD RATIO = 1*16) AS A FUNCTION OF THE NUMBER OF EVENTS (DEAIHS) 
USING A ONE SIDED .05 LEVEL LDGRANK TBST^ 

^Based on Freedntn L.S. (1982) "Tables of the Nunber of Patients Required In Clinical ^rlals Using the Logrank Tos^" 
STATISTICS IK MEDICINE. 1, 121-129. 
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2tt^ number of events represents the deaths in both groups cxi±>ined. FoUown?) only extends through the follou-up 
period of the shorter of the two grcups (eg. for diagnasis yto- 1975 and 1980 with follownm through 1986, only 
deaths within 7 ve<urs count as events.) 
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FIGURE 2 



POWER 
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POWER TO DETECT A DIFFERENCE IN FIVE YEAR SURVIVAL FROM .67 10 .72 
(HAZARD RATIO = 1.21) AS A FUNCnCN OF THE NUMBER OF EVENTS (DEATHS) 
USING A ONE SIDED .05 LEVEL LDGRANK TEST^ 

^^Based on Freeda^n L.S. (1982) •Tables of the Number of Patients Required in Clinical Trials Using the Logrank Tes: 
STATISTICS IN MEDICINE, 1, 121.129. 
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Now page 22. 
Now page 23, 



Now page 23. 



See pages 24-25. 



Page 1^ 



Loo^ ing again at Figures 1 and 2, we would need 1,119 events to 
detect a change -from .672 to .71 with 80 percent power, and we 
would need 685 events to detect a change ^rom .672 to .72 witl) 
80 percent power. 



The Department's analysi 
trials reviewed and ana 1 
comprehensive review o-f 
colleagues, "The E-f-fects 
Therapy on Mortality in 
Randomized Trials among 
survival rises -from 65 . 7 
chemotherapy, a smaller 
Moon et al . review. I -f 
signi-ficant e-f-fect, is i 
analysis would have even 



s described above uses dat 
yzed by Moon and colleague 
ad juvan t trials undertaken 
o-f Adjuvant Tamoxi-fen and 
Early Breast Cancer: An Ov 
30 ,000 Women , *' suggests th 
percen t to 73. 0 percent d 
di "f "ference than that sugge 
this smaller, but statisti 
ndeed the ex pec ted bene-f 1 1 
less O'f a chance o-f detec 



a -from selected 
s. A 
by R. Peto and 
o-f Cytotoxic 
erview o-f 70 
at -five year 
ue to adjuvant 
sted by the 
cally 

, then the 6A0 
ting it. 



ariate survival modeling, 
pproach used by GAO is one 
basic aims o-f the GAO 
-8, IS related to the 
GAO's second analysis. As 

insu-f -f icient power to test 
justing "for covariates 
tionship between the year — 

and survival . However , as 

over time in the 



Page 3-7, Paragraph 2 

Analysis o-f the SEER data using multiv 
o-f which the Cox propor' -nal hazard a 
example, is quite ap* ^riate -for the 
study. The GAO c^n. , shown on page 3 
pairwise year comparisons used in the 
noted above, the pairwise analysis has 
the basic hypotheses o-f the study. Ad 
could improve power i-f there is a rela 
independent variables, the covariates , 
GAO points out, there is little change 
covariates . 

Page 3-9. Paragraph 2 

The GAO statement that it can -find no observable improvement in 
breast cancer patien t surv ival should be amended to speci-f y that 
it cannot -find evidence o-f improvement -for the subset o-f women 
included in its analysis and that its analysis had less than a 
50 percent chance o-f detecting an improvement o-f the magnitude 
ex pec ted -from clinical trials. The current statemen t ignores the 
improvement seen in large c lin ical tria Is . 

Page 3-10 

The -footnote on the bottom o-f the page is a ^ ey point that should 
be emphasized in the text; there are not enough data and the 
sample size is not large enough to detect the di-f -ferences 
expected based on a comprehensive review o-f the reported clinical 
trials . 
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Page \Z 



Deleted. 



Page 3--1K Pafaqraph 



It IS not clear whether the range of 10 to 30 percent survival 
improv/ement refers to relative or absolute percent change m 
survival. There is little evidence to suggest that the survival 
improvement conferred by chemotherapy is larger than 10 percent 
absolute difference at 5 years. 

The Department believes that many cancer patients for whom 
adjuvant chemotherapy has been shown to improve survive 1 in 
clinical trials do not *^eceive it or do not receive the regimens 
most likely to be effective. The Department has jr.Latiple 
programs in place to disseminate information to the public <nnd to 
physicians about cancer prevention, diagnosis, and treatment. 
Efforts to provide treatment information to physicians are 
described in the recent response to the GAO report "Cancer 
Treatment J The National Cancer Institute's Role in Encouraging 
Doctors to Use Breakthroughs." 



GAO expresses a concern that the Department may be promoting 
therapies which, though effective in clinical trials, do not 
change patient outcomes when used in clinical practice. There is 
no inherent reason why adjuvant breast cancer therapies which are 
effective m clinical trials, cannot be del ivered correc t ly in 
clinical practice. he critical concern is that practicing 
physicians may modify effective therapies thereby rendering them 
less Chan optimally effective. 



Third paragraph (first line), the word "most" should be "must". 



The Department believes that before attempting to design a 
patterns of care study, an advisory committee should consider 
whether such a study is feasible, taking into account the 
obstacles enumerated m the G eneral Com ments section. The 
Department believes that if an advisory committee recommends that 
such a s^udy be done, it should be preceded by a pi'ot study to 
assure feasibility prior to undertaking a full-scale oatterns of 
care study. The panel convened should consider the advisability 
and feasibility of a broader study of the delivery of cancer 
therapy, extending beyond breast cancer. 



Deleted. 



Page 3-12. Paragraph 3 and continuing on page 3-13 



Now page 26. 



Page 3-14 



Now page 27. 



Page 3-15 
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Now pages 30-32. 



Page I-l to 1-3 



There are many problems and inconsistencies m this sectio*. To 
summarize: 1) most life table methodologies do not add the one- 
haH interval oi survival, but rather assume everyone survived 
one-hal-f o-f the interval m which they were censored; 2), the 
example is inappropriate; 3) changing the study cut-of-f date and 
assuming that everyone not reported dead, but lost to -follow-up, 
was alive m December 1986 is not justified. The Department 
would use a conservative approach and employ December 1985 as the 
study cut-off rather than December 1986 because follow-up j.s 
fairly complete by this date, i.e. almost all patients are either- 
dead or known to be alive on December 31, 1*^85. GAO's approach 
could bias its results m either direction. 



The two different ways of coding the year variable (the direct 
value or 1 through 11) will produce identical findings. These 
are not really separate analyses. More importantly, the analysis 
done m this manner is not the most appropriate. If year is used 
as a proxy for chemotherapy, then this analysis implies a 
monotonic functional relationship between survival find year. 
This was not the case m the SEER data, as shown m Table 1, and 
this violates the assumption of proportional hazards. 

An alternative is to use the percent receiving chemotherapy each 
diagnosis calendar year as a continuous independent variable. 
Everyone diagnosed m a particular calendar year would have an 
identical value of the covariate. This alternative would be a 
more appropriate way of approaching the data and, m the 
departmental analyses, does yield results suggesting a survival 
benefit from chemotherapy. Assumptions about follow-up, and the 
use of covariates, however, do affect the results of this 
analysis. There are several ways to approach the SEER data, and 
the GAO analysis did not utilize methods most lively to 
demonstrate the expected survival benefit. 



The calculations m the last column of Table I.l imply a decision 
rule that says if at least one of the two pairwise comparisons is 
significant, then one would conclude a significant benefit of 
chemotherapy m the general population. However, this is not the 
decision rule used m the report, since the 1975 versus 1980 
comparison is significant. 

The power estimates reported m Table I.l ^Appendi:; I), according 
to the footnote, make the assumption that no women received 
chemotherapy m 1975 and 1976 and that 100 percent were treated 
with chemotherapy post 1976. Using these assumptions, GAO 



Now page 31. 



Page 1-3 Paragraphs 2 and 3 



Deleted. 
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Page 15 



overestimates the power to detect survival dif -ferences . The 
Departme.it notes that mors appropriate calculations indicate a 
lower power to detect di-f -ferences o-f a magnitude that can be 
expected based on clinical trials data. 
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